Be Brief, And They Shall Learn: Generating Concise Language Feedback for a Computer Tutor

To investigate whether more concise Natural Language feedback improves learning, we developed two Natural Language generators (DIAG-NLP1 and DIAG-NLP2), to provide feedback in an Intelligent Tutoring System that teaches troubleshooting. We systematically evaluated them in a three way comparison that included the original system, which generates overly repetitive feedback. We found that DIAG-NLP2, the generator which intuitively produces the best, corpus-based language, does engender the most learning. Distinguishing features of the more effective feedback are: it obeys Grice’s maxim of brevity, it is more directive and uses a specific type of referring expressions. Interestingly, simpler ways of restructuring the original repetitive feedback as done in DIAG-NLP1, such as exploiting the hierarchical structure of the domain, were not effective. Since the design of interfaces to Intelligent Tutoring Systems often includes verbal feedback, we suggest that: if the number of different contexts in which verbal feedback is provided is high, such feedback should be based on corpus studies, and generated by techniques more sophisticated than template filling.